{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jO8Bc9YcB5RB"
   },
   "source": [
    "<img src='http://www-scf.usc.edu/~ghasemig/images/sharif.png' alt=\"SUT logo\" width=200 height=200 align=left class=\"saturate\" >\n",
    "\n",
    "<br>\n",
    "<font face=\"Times New Roman\">\n",
    "<div dir=ltr align=center>\n",
    "<font color=0F5298 size=7>\n",
    "    ML for Bioinformatics <br>\n",
    "<font color=2565AE size=5>\n",
    "    Computer Engineering Department <br>\n",
    "<font color=3C99D size=5>\n",
    "    Homework 2: Practical - Random Forest and XGboost <br>\n",
    "<font color=696880 size=4>\n",
    "    Bahar Oveisgharan (bahar.oveis.2000@gmail.com) <br>\n",
    "    Ghazal Farahani (farahaniqazal@gmail.com)\n",
    "\n",
    "____\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I5XXN4FcB5RG"
   },
   "source": [
    "### Full Name : \n",
    "### Student Number : \n",
    "__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U2sVgR-sB5RG"
   },
   "source": [
    "### Breasr Cancer\n",
    "Most of us know someone who struggled with breast cancer, or at least heard about the struggles facing patients who are fighting against breast cancer.The most important part of a process of clinical decision-making in patients with cancers, in general, is the accurate estimation of prognosis and survival duration. Breast cancer patients with the same stage of the disease and the same clinical characteristics can have different treatment responses and overall survival.\n",
    "In this practical assignment, you are going to train a Random Forest and XGBoost model on <code>breast_cancer.csv</code> dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nBPdzweDCTgX"
   },
   "source": [
    "Import all the libraries you may need here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cEiXTug2CXEc"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1e_UXHe3B5RH"
   },
   "source": [
    "#### Loading the Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jaRibnlAB5RI"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5zPud3PpB5RJ"
   },
   "source": [
    "### Data Exploration\n",
    "#### Let's start off by exploring the files we just imported. it's not necessary to do any visualization just a statistical summary of the data would be enough. split your data to train and test."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "H0t8uLzpB5RK"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DE7hEApWB5RK"
   },
   "source": [
    "### Data Preparation\n",
    "#### Creating two custom transformers to put on our pipeline:\n",
    "* To split the data into categorical and numerical features and\n",
    "* To preprocess the categorical features. \n",
    "* Create the X feature matrix and the y target vector.\n",
    "* split the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2e66ULhnB5RK"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "icoG9xBNC9Xr"
   },
   "source": [
    "# Implementing two different models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ssH7NCsIB5RL"
   },
   "source": [
    "## Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MP7VAAhcB5RL"
   },
   "source": [
    "### train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "L4KpaB4OB5RL"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TI5qxBKQB5RM"
   },
   "source": [
    "### Model assessment\n",
    "#### Print Train Accuracy, Test Accuracy and classification Report.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zGz27IJvB5RM"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GBI9WYd-B5RM"
   },
   "source": [
    "### Hyperparameter tuning :\n",
    "#### Randomized Search Cross Validation and Grid Search Cross Validation\n",
    "report best hyperparameters in each part."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2FBIJSI-B5RM"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "39cwqDhLB5RM"
   },
   "source": [
    "### Find important features with Random Forest model \n",
    "#### Visualize feature scores of the features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "9G1jOdwGB5RN"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DdulRxFFDfSS"
   },
   "source": [
    "### * Improve Model (Bonus)\n",
    "In this bonus part, you can add your ideas for improving your model's performance. implement it and compare the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IP9y4KlbB5RN"
   },
   "source": [
    "## XGBoost\n",
    "### Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "zlIiW7hVB5RN"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PLFSUhl6B5RN"
   },
   "source": [
    "### Model assessment\n",
    "#### Print Train Accuracy, Test Accuracy and classification Report.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "V51DuD_3B5RN"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nHjv0Ud3B5RO"
   },
   "source": [
    "### Plot the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "kghhayz8B5RO"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JvzcxNzxB5RO"
   },
   "source": [
    "### Hyperparameter tuning: Grid Search and Cross validation\n",
    "* create a default XGBoost classifier.\n",
    "* create the Kfold object. You can use tratifiedKFold from sklearn.model_selection.\n",
    "* create the grid search object. You can use RandomizedSearchCV from sklearn.model_selection.\n",
    "* fit grid search."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7wZQNavnB5RO"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2kSBv8dpB5RO"
   },
   "source": [
    "### Assessing model performance using the best model from grid search\n",
    "#### Print Train Accuracy, Test Accuracy and Classification Report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "sT7AgRtDB5RO"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BNRbwqeFB5RP"
   },
   "source": [
    "### Create the feature importances plot, plot a decision tree from the booster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0uuAhFvPB5RP"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cJkzsT8rDlk5"
   },
   "source": [
    "### * Improve Model (Bonus)\n",
    "In this bonus part, you can add your ideas for improving your model's performance. You can implement your model and compare the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "4RylEYZJDmPq"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparison between XGBoost and Random Forest Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "b2tLkv1FEHKf"
   },
   "source": [
    "Compare the results from these two models. How Would you rate each method in terms of its performance? What's the difference between these models? explain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "pycharm": {
   "stem_cell": {
    "cell_type": "raw",
    "metadata": {
     "collapsed": false
    },
    "source": []
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
